Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

feature(pu/zt): add 2048 env and Stochastic MuZero #64

Merged
merged 32 commits into from
Sep 12, 2023
Merged

feature(pu/zt): add 2048 env and Stochastic MuZero #64

merged 32 commits into from
Sep 12, 2023

Conversation

puyuan1996
Copy link
Collaborator

@puyuan1996 puyuan1996 commented Aug 8, 2023 •
edited
Loading

  • add stochastic muzero ptree and ctree, policy, model
  • add 2048 env and unittest
  • add 2048 bot
  • add 2048 related configs
  • The benchmark results in the 2048 environment:

  • The render video to illustrate the performance of Random action, Bot action, MuZero, and Stochastic MuZero agents in the 2048 environment.

    • Random action (~1000):
      game_2048_random

    • Bot action (~30000-70000):
      game_2048_bot_small

    • MuZero sim=100 (~27956):
      game_2048_muzero_ns100_s0

    • Stochastic MuZero sim=100 (~37356）:
      game_2048_stochastic_muzero_ns100_s0_small

    • Where the term sim represents the number of simulations used for evaluation in MCTS, and the score within brackets corresponds to the score procured by each respective agent.

#41

timothijoe and others added 16 commits June 10, 2023 11:33
@timothijoe
add stochastic mz ptree
27d9b01
@timothijoe
add stochastic mz ctree
3799b46
add box2d, classic conrol, and 2048 config
14e3822
made corrections to the comments and naming issues
06c0558
made corrections to the comments and naming issues
fa88aab
ok
b7a3fba
ok
9168a1b
@timothijoe
ok
648f747
@timothijoe
ok
4693272
@puyuan1996
Merge branch 'main' of https://github.com/opendilab/LightZero into de…
7000eed 
…v-stochastic-mz
@puyuan1996
polish(pu): polish game_2048_env
11b4b7b
@puyuan1996
polish(pu): polish chance encoder
6bd0310
@puyuan1996
Merge branch 'explicit_chance_branch' of https://github.com/timothijo…
a77a5bf 
…e/LightZero into dev-stochastic-mz-true-chance
@puyuan1996
fix(pu): fix chance encoder related loss
f85aec3
@puyuan1996
sync code
b22dea7
@puyuan1996
polish(pu): polish 2048 env, add env save_render_gif method, add 2048…
860fda1 
… env unittest, add stochatic muzero model unittest
@puyuan1996 puyuan1996 added algorithm New algorithm enhancement New feature or request environment New or improved environment labels Aug 8, 2023
@puyuan1996 puyuan1996 mentioned this pull request Aug 8, 2023
Closed
@puyuan1996 puyuan1996 changed the title feature(pu/zt): add 2048 env and related stochastic muzero algo. feature(pu/zt): add 2048 env and Stochastic MuZero Aug 8, 2023
PaParaZz1
PaParaZz1 requested changes Aug 9, 2023
View reviewed changes
zoo/game_2048/config/stochastic_muzero_2048_config.py Outdated Show resolved Hide resolved
zoo/game_2048/config/stochastic_muzero_2048_eval_config.py Outdated Show resolved Hide resolved
zoo/game_2048/config/stochastic_muzero_2048_config.py Outdated Show resolved Hide resolved
zoo/game_2048/config/muzero_2048_config.py Outdated Show resolved Hide resolved
lzero/policy/muzero.py Outdated Show resolved Hide resolved
lzero/mcts/buffer/game_segment.py Outdated Show resolved Hide resolved
@PaParaZz1 PaParaZz1 mentioned this pull request Aug 9, 2023
Closed
PaParaZz1
PaParaZz1 requested changes Sep 7, 2023
View reviewed changes
lzero/policy/muzero.py Outdated Show resolved Hide resolved
lzero/policy/stochastic_muzero.py Outdated Show resolved Hide resolved
lzero/policy/stochastic_muzero.py Show resolved Hide resolved
lzero/policy/stochastic_muzero.py Outdated Show resolved Hide resolved
lzero/policy/utils.py Show resolved Hide resolved
zoo/game_2048/entry/rule_based_2048_config.py Outdated Show resolved Hide resolved
zoo/game_2048/envs/game_2048_env.py Show resolved Hide resolved
PaParaZz1
PaParaZz1 approved these changes Sep 12, 2023
View reviewed changes
zoo/game_2048/config/stochastic_muzero_2048_config.py Show resolved Hide resolved
lzero/mcts/tests/tictactoe_muzero_bot_mode_config_test.py Show resolved Hide resolved
lzero/policy/stochastic_muzero.py Outdated Show resolved Hide resolved
lzero/policy/stochastic_muzero.py Outdated Show resolved Hide resolved
lzero/policy/stochastic_muzero.py Outdated Show resolved Hide resolved
zoo/game_2048/envs/game_2048_env.py Show resolved Hide resolved
lzero/model/stochastic_muzero_model.py Outdated Show resolved Hide resolved
@puyuan1996 puyuan1996 merged commit 9c42878 into main Sep 12, 2023
0 of 6 checks passed
@PaParaZz1 PaParaZz1 deleted the dev-stochastic-mz-true-chance branch October 17, 2023 05:24
@PaParaZz1 PaParaZz1 mentioned this pull request Jun 20, 2024
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@PaParaZz1 PaParaZz1 PaParaZz1 approved these changes

Assignees
No one assigned
Labels
algorithm New algorithm enhancement New feature or request environment New or improved environment
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@puyuan1996 @PaParaZz1 @timothijoe